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FOREWORD 


Policymakers today are uniquely positioned to leverage data to inform decision-making in unprecedented 
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Carolina, in particular, has a rich history of valuing and prioritizing the collection of relevant education data 
and information, giving it a head start on creating a modern statewide longitudinal data system (SLDS). 


But what does a high-quality SLDS look like and how can it support our complex education system? 


We know that learning, and other key factors that influence it such as health and nutrition, begins even 
before birth. And we also know that in today’s rapidly evolving economy, learning must be a continuous 
endeavor that spans an individual’s career. As a result, it is critical that an SLDS incorporates a broad 
array of systems, including early childcare, pre-K, K-12, community colleges, private and public four-year 
postsecondary institutions, and state agencies who own workforce data. 


Through conversations with SLDS leaders across the country, we have seen that a lack of collaboration across 
state agencies, including limited data sharing, poor coordination of data consistency, and lack of investment 
in agency systems can severely limit the success of an SLDS. 


However, policymakers are in the unique position to coalesce these stakeholders to build a system that can 
be sustained across multiple administrations. 


Additionally, legislators need to ensure that system governance is well structured, building trust and 
transparency among and between partners. The governing body serves a pivotal role in ensuring the SLDS is 
functional. Building a governance structure that is transparent, codified in legislation, and fully funded will 
ensure that the system functions efficiently and is flexible enough to grow over time to meet the evolving 
needs of the state. 


As we move forward, we need to think critically about how to use the data system in a way that informs 
policy decisions. How, for example, can district leaders use this system to inform their work? How can 
institutions use this information to improve educator preparation programs? 


There are countless ways that a high-quality system can be leveraged to improve outcomes for all young 
people. We look forward to helping drive progress on this important work. 


Javaid Siddiqi, Ph.D. 
President & CEO 


GUIDING QUESTIONS FOR POLICYMAKERS 


This report explores the history, purpose, and future of statewide longitudinal data systems. While the 
focus of this report is on North Carolina, it also draws on national lessons from other states, especially 
Kentucky and Washington. North Carolina has already made significant progress on building out its data 
systems but will require additional work in order to meet the education and research needs of the state. 


As policymakers take a closer look at statewide longitudinal data systems (SLDS), there are a number of 
questions they should consider when determining the effectiveness and functionality of the Systems. The 
following questions should help guide policymakers as they consider ways in which they can support the 
creation and use of education data systems. 
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What questions do we need to answer in order to make progress toward our state’s 
education goals? 

Many of the questions that can help inform and shape policy decisions can only be answered by combining 
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Which audiences will benefit from data provided through the System? 
A system should serve policymakers, state-level agencies, system partners, researchers, and families. 


What partners are engaged in data sharing? 

Policymakers should consider whether all educational entities are sharing data with the SLDS, including early 
intervention, early childcare, pre-K, K-12, community colleges, private and public four-year postsecondary 
institutions, and state agencies who own workforce data. 


Does the SLDS have a robust governance structure in place to provide necessary 
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The System needs a clear governing body, codified in legislation, which has worked closely with partners in 
order to build and operate a system that produces timely and accurate data, while ensuring that student 
privacy is protected. 


What are the staffing and technological requirements needed to reliably collect, 
maintain, and submit data? What personnel are needed to ensure effective 
governance and communication? 

In order to function effectively, each partner entity must have sufficient human capital to submit, clean, 
and analyze data. Additionally, the governing body must have analysts to complete quality control checks, 
process requests, and coordinate partners. 


How is the System funded, and is that funding sustainable? 
The System must be funded through recurring funds from the legislature. Additionally, each partner agency 
must have sufficient recurring funds to support infrastructure and human capital needs. 


How will stakeholders access the information in the System? 

The System should produce sets of de-identified data that can be analyzed by researchers, regular 
aggregate reports to serve policymakers, and a public-facing interface with data presented in a way that 
is easily understood. 


What steps are being taken to ensure that data privacy is protected? 
Efforts to ensure data privacy must be at the forefront of all statewide data efforts. 


CONNECTING THE CONTINUUM 


Longitudinal Data Systems in North Carolina 


01 INTRODUCTION 
07 STATEWIDE LONGITUDINAL DATA SYSTEMS 


01 History & Funding 
03 Structure and Organization 
05 Key Characteristics of a High-Quality SLDS 


06 LANDSCAPE ANALYSIS: 
NORTH CAROLINA 


06 A Strong History of Data 

08 The Creation of the NCSchoolWorks Longitudinal Data System 

10 North Carolina Education Longitudinal Data System 

11 Challenges Facing the North Carolina Education Longitudinal 
Data System 

11 Future of Longitudinal Data Systems in North Carolina 


12 STATE SLDS CASE STUDIES: 


KENTUCKY & WASHINGTON 
12 Kentucky 15 Washington 
12 History and Formation 15 History and Formation 
12 System Overview 15 System Overview 
13 Governance 15 Governance 
13 Use and Impact 16 Use and Impact 
14 Next Steps 16 Next Steps 
14 Keys to Success 17 Keys to Success 
14 Challenges 17 Challenges 


18 GUIDING QUESTIONS FOR POLICYMAKERS 
20 GLOSSARY 
21 CITATIONS 


Introduction 


The rise of data collection in the public and private sectors has presented policymakers with an unprecedented 
opportunity to leverage vast amounts of data to create more efficient systems, improve program performance, 
and drive equitable outcomes. Empowering policymakers and practitioners with functional access to accurate 
data has the potential to change the way we deliver education. States across the country are currently seeking 
ways to build systems that effectively collect, compile, and analyze these data to answer complex questions. 


With a strong history of developing and utilizing data systems that span the fields of education, health, and the 
workforce, North Carolina could lead the nation in the development and use of longitudinal data systems. Through 
the strong commitment and relentless efforts of the North Carolina Department of Commerce, North Carolina 
Community College System (NCCCS), North Carolina Department of Health and Human Services (NCDHHS), North 
Carolina Independent Colleges and Universities (NCICU), North Carolina Department of Public Instruction (NCDPI), 
and the University of North Carolina System (UNC), significant progress has been made in creating longitudinal 
data systems in North Carolina. However, North Carolina continues to struggle to connect disparate data sources 
to one another and ensure consistency across systems, keeping the state from fully leveraging data to regularly 
inform policy and practice in a meaningful way. 


Most state agencies collect data independently of one another and typically do not have structures in place to 
guarantee alignment with the types of data points collected and how each are defined. Though each agency has 
some capacity to use its own data to analyze performance, larger policy questions can be answered when the 
data are compiled across agencies and time. These can be questions of practice that guide the work of principals 
and teachers, such as “What percentage of my secondary students required remedial coursework in my subject 
area at the postsecondary level?” They may also be questions of policy that impact broader programmatic and 
budget decisions, such as “How likely are students who are reading on grade level by third grade to complete 

a postsecondary degree or credential?” To answer these questions, an analyst needs student-level data from 
multiple government agencies over a number of years—something that has proven to be challenging in most 
states. 


As data collection has grown in prominence, privacy concerns have also risen. Along with the benefits that come 
with this data collection—such as the personalization of services and greater efficiencies—many constituents are 
concerned about privacy, and policymakers need to ensure any data systems are built with security in mind. 


Statewide Longitudinal Data Systems 
History & Funding 


The first major push for statewide longitudinal 

data systems (SLDS) came through the Educational 
Technology and Assistance Act of 2002, which created 
the first competitive federal grant process to support 
state education agencies in the development of these 
systems.' These grants were first distributed by the 
Institute of Education Sciences (IES) in 2006, with 
subsequent grants being awarded in 2007, 2009, 2012, 
and 2015. The largest block of grants was awarded as 
a part of the American Recovery & Reinvestment Act 
of 2009 (ARRA), which supplemented the 2009 grants 
with an additional $230 million. In total, over $700 
million has been awarded to 47 states, the District 

of Columbia, Puerto Rico, and the U.S. Virgin Islands 
through the SLDS grants. 


Additional grant funding has been awarded through 
the Race to the Top competitive grant process, 


which encouraged states to make policy reforms that 
would enhance standards and assessments, improve 
the collection and use of data, increase teacher 
effectiveness, and turn around struggling schools. 
More specifically, ARRA required that states wishing 

to receive Race to the Top funds must have made 
significant progress in establishing longitudinal data 
systems, making this another mechanism to incentivize 
states to improve their data systems. 


Figure 1 | SLDS Federal Grant Funding by Year? 
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Figure 2 | SLDS Federal Grant Program Funding by State® 
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Structure and Organization 


Statewide longitudinal data systems connect individual-level data over time and across agencies. While most 
states refer to these systems as SLDSs, in North Carolina the System is known as the Education Longitudinal Data 
System (ELDS). These data systems allow researchers and policymakers to follow a student’s academic career, draw 
inferences about what types of programs and services are working well, and identify disparities in performance 
among subgroups. Education-focused SLDSs are often referred to as P-20W systems, which cover pre-Kindergarten 
(P), K-12 and postsecondary (20), and workforce (W) data. 


All 50 states and the District of Columbia are able to connect data between systems in some capacity, and 38 
states and the District of Columbia are able to connect at least two of the four core systems (early learning, K-12, 
postsecondary, and workforce). Only 17 states and the District of Columbia have developed a full P-20W system that 


connects all four core systems.* 


There are two main types of SLDSs— 
centralized and federated—though, 
in practice, there are many variations 
that fall somewhere in between the 
two approaches. North Carolina’s 
system, for example, will pull data 
from both federated and centralized 
systems. Each type has the capacity 
to collect the same data and facilitate 
the same types of analyses, but they 
differ in how the data are compiled 
and where the databases are housed. 
According to recent analysis, only 11 
of the 38 systems are federated.° 


Data are typically matched across 
systems based on a set of student 
characteristics, including name, date 
of birth, gender, other demographic 
data, and, in some instances, social 
security numbers.* Many states 

also assign a unique identifier, or 
UID, to each student, which follows 
them throughout their academic 
career. The use of a UID ensures that 
student data are not associated with 
personally identifiable information 
once in the system. This UID is 
consistent across systems and would, 
for example, follow a student as they 
move from early childhood services 
to K-12, community college, and 
university. UIDs can be especially 
helpful in a federated system, 

as they can facilitate quick and 
accurate matching across systems. 
The consistent and accurate 
assignment of a UID is critical to the 
success of systems that use them. 


Figure 3 | Centralized vs Federated Data Systems 
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and approved based on pre-determined agreements. 


Figure 4 | Data That Can Be Collected in a North Carolina P-20W System 


Most states and state agencies are already collecting a wealth of information on students, including the more 
detailed data points listed below, many of which are specific to the North Carolina system. A robust P-20W 
system would cover a broad range of student, teacher, and school characteristics, as well as data on student 
performance from agencies—starting with early childhood programs and services going through workforce 
outcomes. Consistent and uniform reporting on each of the variables will allow for reliably connecting these 
data for meaningful analysis. 
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Figure 5 | America COMPETES Act: 12 Essential Elements of an SLDS by Academic Level 


In 2007, the America COMPETES Act called for alignment across the education continuum, including 
improvement of longitudinal data systems. The Act identified the following 12 essential elements of an SLDS.’ 
These data points range from student achievement and demographics to remedial coursework and the ability to 
match students with teachers. 
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Key Characteristics of a High-Quality SLDS 


Statewide longitudinal data systems must possess a number of characteristics and require a favorable 
policymaking environment in order to provide high-quality, useful education data to a wide variety of 
stakeholders. Among other characteristics, the following outlines some of the most critical elements to a 
successful SLDS. 


> 


Widespread Policymaker Support: In order to effectively drive change, advocates for improving a 
state’s education data system must build the momentum needed to develop support among elected 
state officials and state agency staff. State legislatures have the capacity to codify SLDS policy in ways 
that can sustain a system across multiple administrations, and the governor can serve as a coordinating 
force. Policymakers are in the unique position to have the leverage necessary to compel collaboration. 
Meaningful Agency Collaboration: Many SLDSs have struggled due to lack of collaboration across 
state agencies. This comes in many forms, including limited data sharing, poor coordination of data 
consistency, and lack of investment in agency systems. It is critical that all agencies see the potential 
benefits of the System and that they are committed to transparency and building trust. 

A High-Quality Governance Structure: Governance structures are responsible for making critical 
decisions about how student data are collected, linked, reported, and protected. In addition, SLDSs 
require a significant amount of maintenance and ongoing updates. To keep the System functioning, 

it is critical that structures are set up to ensure regular communication. 


Robust Privacy and Security Protocols: Concerns about privacy are shared by a variety of stakeholders, 
including parents, schools, agencies, and policymakers. These concerns are justifiable but can be 
properly addressed using the appropriate protections in data management, such as de-identification 
and data minimization. 


Standardization of Data: With multiple agencies gathering data, it is critical that there is a common 
understanding of how each variable should be defined. Small differences in definitions can make 
analyses unreliable and tasks burdensome for researchers. Agencies need to build their capacity to 
ensure the reliability of their own data and coordinate with other agencies to ensure accurate data 
matching. 


Adequate Technical Infrastructure & Dedicated Human Capital: Each SLDS partner must have 
sufficient staff and technology to allow for efficient and accurate data recording and transferring. 
Many states have failed to supplement federal start-up money with recurring state funds in order to 
maintain systems, staff, and technological capacities. 

Sustainable Internal Leadership: The institutional knowledge possessed by staff is valuable, and the 
loss of that knowledge due to staff transitions can damage the System as a whole. Too often, agencies 
rely on one individual to lead the work, build the requisite knowledge, and develop the necessary 
relationships. Agencies need to be thoughtful about structuring this work so that it is not a challenge to 
sustain it in the wake of staff turnover or after electoral transitions. 
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Landscape Analysis: North Carolina 


A Strong History of Data 

North Carolina’s education system has long served as a resource for educational research due in large part 

to the presence of data systems capturing student, teacher, and school performance data going back to the 
mid-1990s. North Carolina first created a longitudinal data system with the launch of the Common Follow-up 
System (CFS) in 1992. Originally created by state agencies to capture outcomes of public education, employ- 
ment, and training programs, the CFS was written into statute by the North Carolina General Assembly in 1995.8 
The CFS currently includes data from the North Carolina Department of Commerce, North Carolina Department 
of Health and Human Services (NCDHHS), North Carolina Department of Public Instruction (NCDPI), North 
Carolina Department of Public Safety, North Carolina Community College System (NCCCS), and the 17-campus 
University of North Carolina System (UNC).? 


In 2000, the North Carolina Education Research Data Center was established through a partnership between 
NCDPI and the Duke University Center for Child and Family Policy.*°? Researchers from nonprofit and universi- 

ty settings, as well as government agencies, can pay to access more than 20 years of data on North Carolina’s 
public schools, students, and teachers. In addition to this private source of educational data, the state has a long 
history of publicly available workforce and postsecondary data as well. This commitment to gathering data laid 
the groundwork for what would eventually become North Carolina’s Education Longitudinal Data System (ELDS). 


In an effort to create the infrastructure needed to analyze different types of data over time and improve the 
quality and accessibility of pre-K through secondary educational data, NCDPI applied for and received a $6 
million grant from the U.S. Department of Education as part of the Statewide Longitudinal Data Systems Grant 
Program in 2007. Under this grant, NCDPI developed the Common Education Data Analysis and Reporting 


Figure 6 | The North Carolina Education Longitudinal Data System 


NORTH CAROLINA EDUCATION LONGITUDINAL DATA SYSTEM (ELDS) 
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*Data from North Carolina’s independent colleges and universities are not currently available for requests pending the 
creation of formal rules and memoranda of understanding between each independent college or university and the partner 
organizations. 
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System (CEDARS), which included the creation of a 
Unique Statewide Identifier (UID) for both students 
and teachers. Use of a UID allowed for staff- and 
student-level data to be matched across the K-12 
continuum. 


The effectiveness of CEDARS was supported by a 2010 
statewide mandate from NCDPI requiring all North 
Carolina public schools to utilize the PowerSchool 
technology platform as an operational data store. 

Of critical importance, PowerSchool utilizes the UID 
interface for students and teachers, improving data 
validity for the K-12 sector. 


A student enrolled in a North Carolina K-12 public 
school is assigned a UID upon enrollment. This is a 
mandatory step in the school enrollment process 
across K-12 schools in the state. Ensuring that a 
student’s UID successfully travels with them to a 
postsecondary setting has proven to be a challenge. 
Part of the difficulty lies in the timing of when an 
institution checks to determine if a student already 
has a UID. Students transferring between K-12 
schools are generally matched or assigned new UIDs 
without issue because all North Carolina schools 
match UIDs locally (i.e., in person) and use the same 
technology platform for enrollment across schools. 


When a student enrolls in a school for the first time, 
the System automatically checks to determine if that 
student has a UID and either matches the pre-existing 
UID to that student or notifies the individual 
completing enrollment that there is a near match (i.e., 
some of the demographic details indicate a match, 
but others do not). In cases of a near match, the use 
of local UID matching allows the near match to be 
resolved immediately, in person, by clarifying any 
inconsistencies in the student’s profile. In instances 
where the student has never been enrolled, a new UID 
is assigned in real-time. 


In 2011, North Carolina was awarded a $6.9 million 
Race to the Top — Early Learning Challenge Grant 
that included the creation of the North Carolina 
Early Childhood Integrated Data System (NCECIDS). 
This birth-to-five system is administered by the 
NCDHHS and includes data on early childhood 
education, health, and social services.** Importantly, 
NCECIDS utilizes the UID software, making it possible 
to integrate early childhood data into the K-12 
longitudinal data system. The North Carolina General 
Assembly has approved recurring state funding to 
support the operation and maintenance of NCECIDS. 


Figure 7 | Examples of Unique Identifier (UID) Assignment and Use 
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NCDHHS is currently working to increase the data 
sources and data points available through NCECIDS. 
Data from the Temporary Assistance for Needy 
Families (TANF) program are anticipated to be fully 
integrated into NCECIDS by the end of 2019. A pilot 
program to integrate data regarding children enrolled 
in Head Start is currently in progress for a subset of 
grantees in preparation for all Head Start data to be 
included. Finally, NCDHHS is working to standardize 
data collection from home visiting programs in 
preparation for those data to be integrated into 
NCECIDS in the future. 


The Creation of the NCSchoolWorks 
Longitudinal Data System 


With the creation of early childhood and K-12 
longitudinal data systems, and the continued use of 
the CFS capturing postsecondary and workforce data, 
North Carolina was primed to create a P-20W system 
that would span each pre-existing system. NCDPI 

led this effort and was awarded a $3.6 million grant 
from the U.S. Department of Education to develop a 
federated, P-20W statewide longitudinal data system. 
The system would later be named NCSchoolWorks. 
The P-20W SLDS collaboration includes NCDPI, UNC, 
NCCCS, NCICU, and the North Carolina Department of 
Commerce Division of Employment Security. 


The efforts of the partner entities were signed into 
law in 2012 with the passage of House Bill 964, creat- 
ing the North Carolina Longitudinal Data System. The 
enacted bill became North Carolina General Statue 
Chapter 116E. Per the bill, the System “is a statewide 
data system that contains individual-level student and 
workforce data from all levels of education and the 
State’s workforce” located administratively within, 
but independently of, NCDPI.*? This statute has been 
used to provide guidance to both NCSchoolWorks 
and the ELDS, which will be discussed later in this 
section. 


The system was originally governed by an 18-member 
North Carolina Longitudinal Data System Board which 
included the agency leads from the partners, as 

well as representatives appointed by the legislature. 
The state statute was amended in 2016 to transfer 
responsibility over the System from the Board to 
North Carolina’s Governmental Data Analytics Center 
(GDAC).*8 Those responsibilities include development 


of an implementation plan, provision of general 
oversight and direction, approval of an annual 
budget, and assurance that privacy and data security 
are in line with other laws. Additionally, an advisory 
committee is to advise the GDAC on data quality 

and data validity. While membership is not included 
in the statute, the advisory committee consists of 
representatives of each of the partner agencies, as 
well as GDAC, and the Office of the Governor. 


The statute goes on to outline the duties and 
functions of the System, which include serving as 

a data broker, compliance with privacy laws and 
policies, and facilitation of data requests for state and 
federal education reporting and public information 
requests. The statute mandates that data accessed 
through the System must only be accessed by 
authorized staff, must be de-identified, and can only 
be used in aggregate form when used in reports or 
when shared as part of a data request. 


By the end of the grant period in 2017, the System 
was found to have met the objectives outlined in the 
grant. 


The System is currently being tested, and while it is 
reported to work, it is only accessible to the partners. 
At present, the system is able to produce data from 
NCDPI, the UNC System, and the Department of 
Commerce. It is possible to match data from NCCCS 
that is pulled from the CFS and there continues to 

be efforts to improve infrastructure at NCCCS such 
that those data can be pulled directly from the 
NCSchoolWorks. Early childhood data from NCECIDS 
cannot be joined to the NCSchoolWorks at this time. 
Additionally, data from North Carolina’s independent 
colleges and universities are not currently available 
for requests pending the creation of formal rules 
and memoranda of understanding between each 
independent college or university and the partner 
organizations. 


In 2018, North Carolina Governor Roy Cooper 
reconvened the North Carolina Education Cabinet, 
which includes the State Superintendent of Public 
Instruction, the President of NCCCS, the President 

of the UNC System, the President of NCICU, 

the Secretary of NCDHHS, and the Secretary of 
Commerce. The Governor charged the Cabinet with 
convening a working group from the NCSchoolWorks 
partners and GDAC to continue the development of 
the broader ELDS with GDAC. 
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Figure 8 | Proposed NCSchoolWorks Data Request Process 
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The Cabinet worked together to formulate a number 
of questions they hoped the System would be able to 
answer, with the goal of turning the data into material 
that could be used to inform decision-making. In 
2019, the Cabinet made a request of NCSchoolWorks 
to track outcomes of graduates of public high 
schools.** The System successfully provided data 
from NCDPI, the UNC System, and the North Carolina 
Department of Commerce. The System was unable 
to retrieve data from the NCCCS; these data were 
accessed via the CFS. 


Once the data were received from each partner, 
students were matched by UID. A social security 
number crosswalk was required to match data from 
the North Carolina Department of Commerce and 
NCCCS as the CFS uses social security numbers as the 
primary identifier. While the request was completed, 
NCSchoolWorks was not able to produce the data 
necessary to complete the request without matching 
data from the CFS. 


North Carolina Education 
Longitudinal Data System 


The ELDS is comprised of NCECIDS, NCSchoolWorks, 
and the CFS. Requests made to this “system of 
systems” will be completed by matching centralized 
data from the CFS to federated data from NCECIDS 
and NCSchoolWorks. North Carolina General Statute 
Chapter 116E authorizes an ELDS that is charged with 
facilitating the exchange of individual-level student 
and workforce data.*® The statute states that the 
System operates independent of NCDPI and GDAC 
but is administratively located within NCDPI. GDAC is 
tasked with overseeing the drafting and adoption of 
rules, providing general oversight and direction to the 
System, approving an annual budget, establishing an 
advisory committee on data quality, and developing a 
plan to establish and operate the System. 


The ELDS Working Group of the Governor’s Education 
Cabinet has developed a number of key principles 
regarding system governance, including that each 
partner will maintain authority over their data, be 
able to request data from other partners in the 
System, and have the right to deny a data request.*® 
Additionally, data shared between partners will be 
identifiable, but data shared outside of the partners 
must be de-identified. 


GDAC and the Education Cabinet continue to move 
the work of the longitudinal data system forward. 
The System contributors are currently reviewing 

a “Contributor Data Sharing Agreement,” which 
will be able to incorporate future partners as well. 
The statute also requires that rules be adopted; 
these rules have been drafted and will be formally 
considered by the North Carolina Rules Review 
Committee. 


The Working Group has recommended to the 
Education Cabinet that an independent consultant 
be engaged, with the charge of working with GDAC 
to develop a plan to improve interoperability and 
modernize the ELDS.” The consultant will work with 
each partner, other key stakeholders, agency leaders, 
and data users to get feedback on the System. The 
consultant is expected to produce a report by early 
2020, which will describe the current state of the 
System, a vision, objectives, and requirements of the 
ELDS, recommendations regarding governance, and a 
sustainability plan. 
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Challenges Facing the North Carolina Education Longitudinal Data System 


> Accurately assigning and matching students to UIDs. UIDs play a critical role in the success of the 
SLDS, particularly in federated systems. Partners in the System report varying degrees of success 
managing this process. If a large percentage of students are not assigned a UID, or if a student is not 
successfully matched to their UID and is assigned multiple UIDs over the course of their academic and 
workforce transitions, the data produced by the System will be compromised. In order to accurately 
assign UIDs, each partner entity must have sufficient resources, including both personnel and 
technology, to manage matching, assigning, and resolving near matches. These resources were found 
to vary across partners. Postsecondary institutions complete the UID match and assignment process 
centrally at their respective system office at set intervals rather than locally at an individual campus in 
real-time. This can cause delays in matching and make it cumbersome to resolve near matches. 


Vv 


Lack of support among partners. While all NCSchoolWorks partner entities expressed an underlying 
belief in the value of the data system, there are differences across partners in perceived value of 

the System to their institution. In some ways, participation in the SLDS was an unfunded mandate. 
Additionally, a number of the partners engage in bilateral data sharing, resulting in questions regarding 
what additional value NCSchoolWorks brings to their work. This variability, paired with a lack of clear 
ownership over the System and internal willingness to commit to data quality, has impacted the progress 
of the partnership. 


Vv 


Lack of vision about how the System should be structured. It is unrealistic to expect the System 
to provide an answer to every question posed. By establishing sets of questions that policymakers, 
researchers, and institutions want answers to, it would be possible to better cultivate the data from 
each institution and potentially create standard reports of aggregate data. 


Vv 


No single responsible party to define data points, ensure data quality control, or support appropri- 
ate data use. Partner entities use the Common Education Data Standards (CEDS), and the user interface 
allows researchers to select data for requests using the CEDS labels and definitions. However, there is no 
process to ensure that each partner is interpreting elements of the CEDS data dictionary the same way. 
Because each partner independently determines which data map to each standard, it is possible that 
these data are not accurately aligned. 


It is equally important that someone who knows the data is able to serve as a resource to potential 
researchers and ensure accurate collection, matching, and use of the data. It is feasible that a data 

analyst role could be created at GDAC, or that this role could be provided by a third-party group, to 
ensure that data are being reported accurately. 


Vv 


Confusion about which entity is responsible for sharing data. An increase in partnerships between 
K-12 and community colleges to support students who participate in dual enrollment and early college 
high school programs can make it difficult to determine which entity is responsible for sharing specific 
data points. Similarly, if a student transfers mid-year, there is a lack of clarity on which partner would 
report that student’s data for that year. 


Future of Longitudinal Data Systems in North Carolina 


Great progress has been made in building a solid foundation for longitudinal data systems in North Carolina. 
The contributing partners in the ELDS Working Group are already having conversations about what the next 
iteration of the data system might be and are thinking ahead to envision what a modern system would look like. 
There is hope that the ELDS would have a public-facing data dashboard where standard data sets are presented 
ina meaningful and accessible way. The Community College System is currently working to enhance its UID 
assignment and matching processes, which would improve the accessibility, accuracy, and timeliness of data 
they submit to NCSchoolWorks and the ELDS. Questions remain about long-term governance of both systems, 
though it is evident that a clear governance structure is necessary to continue driving the work forward. 
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State SLDS Case Studies 


While no two statewide longitudinal data systems are identical, there are certainly lessons that can be 
learned in taking a closer look at how other states have created, legislated, and funded education data 
systems. Kentucky and Washington both offer examples of SLDSs that successfully provide longitudinal data 
which informs legislation and policy in their states. 


Kentucky 
History and Formation 


In 2005, the Kentucky Department of Education (KDE) 
spearheaded the development of a statewide K-12 student 
longitudinal data system in response to being awarded 

a $5.8 million U.S. Department of Education Institute of 
Education Sciences (IES) SLDS grant. A vendor partner was 
selected and charged with the task of designing, develop- 
ing, testing, and implementing the System.*® The Kentucky 
Longitudinal Data System (KLDS) was first implemented in 
mid-2007 as a pilot program and included a limited number 
of districts. 


The System was awarded a second SLDS grant from IES in March 2009 in order to expand the K-12 data 
available in the System. This grant also allowed for the creation of a “P20 Shared Repository,” which linked 
K-12, postsecondary, and teacher certification data.*? Subsequently, the Kentucky P20 Data Collaborative was 
formed, which included representatives from the KDE, the Education Professional Standards Board, and the 
Council on Postsecondary Education. 


The fallout from the national economic recession in 2008 resulted in the Kentucky state legislature 
terminating state funding to the SLDS. The KLDS itself lost $2.2 million in annual funding; additionally, the 
Kentucky Student Information System lost $7 million in annual funding. This loss of funding resulted in a 
reorganization of the System in 2010. In this transition, the P20 Collaborative and its original intent to serve as 
a single data warehouse changed to a “focus on linking the capabilities of multiple systems.” 7° 


Governor Steve Beshear (2007-2015) was a strong supporter of the state’s ongoing data work and signed an 
executive order creating the Kentucky Center for Education and Workforce Statistics (KCEWS) in June 2012 to 
house the KLDS. The KCEWS was ratified into law during the 2013 legislative session. This Center is now called 
Kentucky Center for Statistics (KYSTATS) and is charged with managing the centralized KLDS, collecting data 
for the System, and completing data analysis and reports. The System includes data from K-12, postsecondary, 
and the workforce. 


System Overview 


The KLDS is a centralized data system managed by KYSTATS, an independent office within the Education 
and Workforce Development Cabinet. Of the nearly 40 staff members—including analysts, researchers, and 
developers—nearly half of the staff are working on the KLDS. 


De-identified data are sent to a warehouse from a number of sources including early childhood programs, 
the K-12 system, postsecondary institutions, and workforce partners. The System relies on an algorithm 

to link and match the records using social security numbers and other personal identifiers to match K-12 
data to postsecondary and workforce data. Personally identifiable information is replaced with unique 
system identifiers before moving into the de-identified warehouse. Only the Development Team at KYSTATS 
is engaged in this matching process to ensure data privacy. There are currently 10 years of data in the 
System, with a delay of approximately six months in real-time data availability. Data are made available 
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to policymakers, superintendents, researchers, 

and the public through aggregate reports and in 
response to specific requests. KYSTATS also provides 
online dashboards on common topics, such as 

local workforce areas and academic performance 
proficiency. 


Governance 


The center is governed by a board that consists of 
representatives from the Education and Workforce 
Cabinet, Council on Postsecondary Education, KDE, 
and the Kentucky Higher Education Assistance 
Authority. Effective July 1, 2019, the Cabinet for 
Health and Family Services will join the board. Each 
board member recommends an individual from their 
agency to serve on the Research Committee, which 
is charged with setting a biannual research agenda 
for the System. This has served as an avenue for each 
agency to maintain a sense of ownership of their data 
as they provide direction about how the data will be 
used. 


Figure 9 | Overview of the Kentucky Longitudinal 
Data System?* 
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Use and Impact 


There are several instances in which longitudinal 
data made possible through the KLDS have resulted 
in direct policy impact through increased funding to 
relevant agencies and programs. 


> The statewide initiative, AdvanceKentucky, 
assesses the impact of AP enrollment and success 
on student outcomes.*? Results gathered using 
KYSTATS found that students who completed AP 
exams and earned a qualifying score were more 
likely to receive higher college success metrics 
compared to similar students who did not take 
an AP exam.?3 


Figure 10 | Example of Graphic Available Via 
KYSTATS Data Dashboard” 


Demographic Group 


County Year Choose 3rd Grade T.. 
KENTUCKY 2015 — Gender All Math 


Ready with 


Enrichments 4,247 Distinguished 


Proficient 


Apprentice 


Without a 3rd Grade 
Score 


Without a 3rd Grade 
Score 


> SummerWorks is a program that partners 
with local businesses, nonprofits, government 
agencies, and local community organizations 
to connect youth aged 16-21 with summer 
employment.?° Data from KYSTATS were used 
to evaluate the correlation between working 
during the summer and postsecondary/ 
workforce outcomes for students aged 
16-18 and recent stop outs (i.e., students 
who attended some higher education but 
then stopped attending before completing 
a degree) aged 19-21.*° The study found 
increased employment retention relative to 
other like-students and some higher results in 
postsecondary enrollment for stop outs. 


Vv 


The Kentucky Career and Technical Education 
Feedback Report is compiled by KYSTATS and 
used by the Office of Career and Technical 
Education to create career pathways that 

align to regional workforce needs.*” KYSTATS 
compiled occupational projections for the 
next five years for key sectors defined by the 
Kentucky Workforce Innovation Board. Using 
these projections, the Office of Career and 
Technical Education created the pathways that 
align to these jobs. KYSTATS then produced a 
report to show the alignment so that CTE can 
help schools incentivize high-demand pathways. 
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Next Steps 


KYSTATS continues to work to add additional data to the System. This includes driver’s license information 
to support data matching, corrections, Medicaid, Supplemental Nutrition Assistance Program (SNAP), TANF, 
job projection data, and a number of workforce data sets including nursing licensure, apprenticeships, and 
industry certifications. 


Keys to Success 
A number of factors have contributed to the success of the Kentucky Longitudinal Data System. 


> Use of a centralized data system ensures that stakeholders consistently have access to high-quality 
data that can be pulled in a short amount of time. The centralized system makes it possible for internal 
and external researchers to continue to build on data sets rather than having to request and complete a 
new search as research questions evolve. When compared to a federated system, the centralized system 
ensures that data quality is consistent, as data are cleaned once upon submission to the System, rather 
than having to be cleaned with each submission following a request. 


> There is strong partner engagement and support. 


>» Buy-in across partner agencies has been made possible by the identification of champions of the 
SLDS in each agency. 


>» The System partners are mandated in legislation, making expectations for participation clear. 


>» Astrong data governance system ensures that partner agencies maintain a sense of ownership of 
their data. This sense of ownership is further promoted by the required input and sign-off from 
each partner before a report is released to the public. 


>» The creation of tailored reports to meet the individual needs of an agency, which address specific 
questions that can only be answered with a longitudinal data system, has allowed partners to see 
direct benefits from participating in the System. 


> By publicly publishing a biannual research agenda, KYSTATS promotes trust and transparency in the 
use of the data, while also showing what is possible as a result of the data being shared. 


> By initially piloting the data system in a limited number of districts, the KDE was able to closely 
collaborate with offices within the agency to troubleshoot any challenges regarding data sources 
and matching. Additionally, the limited number of data requests in this early period allowed for the 
standardization of the data itself, as well as collection and storage processes. 


Challenges 


> While KYSTATS receives approximately 30 data requests per month, they continue to be underutilized 
by policymakers in the legislature. KYSTATS is currently developing a plan to engage directly with 
legislators, demonstrating the types of reports and data available through the System in order to 
promote legislative understanding and use of the System. 


> The KLDS has long been reliant on federal grants to support the System and KYSTATS. Since the first 
SLDS grants were distributed by the U.S. Department of Education in 2006, Kentucky has received nearly 
$19 million in funding. Currently, 88 percent of the center’s $4.3 million budget is funded by federal 
grants. The remaining portion of their budget comes from state funding, as well as some cost recovery 
from the completion of external requests. 
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Washington 
History and Formation 


In 2005, Washington Governor Christine Gregoire (2005-2013) 
convened the Washington Learns blue ribbon commission 

to review the state’s education system and make 
recommendations for improvements across the education 
continuum. One of the recommendations of the commission 
was to create a P-20 Council that would, among other things, 
work to develop a longitudinal student data system.”° 

The Council was created by Executive Order in 2007 and was 
charged with tracking student outcomes as they transitioned 
across the education continuum. 


In 2007, the Washington Legislature also passed legislation creating the Education Research and Data Center 
(ERDC).?? The Center was placed under the Office of Financial Management (OFM) and partnered with 

the Legislative Evaluation and Accountability Program Committee to conduct analyses of programs across 
P-20 and the workforce. The legislation required all state agencies involved in education, as well as public 
postsecondary institutions, to create data-sharing agreements with the ERDC. 


In 2009, ERDC was awarded a U.S. Department of Education IES grant to lead the state’s P-20W governance, 
build a data warehouse, and produce analysis based on critical research and policy questions. This grant also 
allowed ERDC to expand the System to include early childhood data. 


System Overview 


The ERDC has a staff of 10 analysts and researchers, as well as additional technical and IT support staff. 
These positions are funded through a combination of recurring state funds and grant funds from SLDS and 
Workforce Data Quality Initiative (WDQI) grants. The Center partners with 11 agencies and organizations 
to offer data from early childhood through the workforce. There are currently 14 years of high-quality data 
available in the System. 


Data are matched using personal identifiers including name, date of birth, gender, and race. The System offers 
separate processes to complete data requests depending on whether or not the request requires re-disclosure 
of personally identifiable information. In an effort to build improved transparency and data ownership, ERDC 
is currently amending its data request process such that partner agencies will be able to opt in to providing 
data to a given request, whereas previously they had to opt out if they did not want their data to be used. 

The turnaround time for completing requests ranges from hours to weeks, depending on the complexity of 
the request and whether pre-existing data sets can be used. The data request process as it currently exists is 
outlined in Figure 11 on the next page. 


Governance 


ERDC differs from other statewide longitudinal systems in that it is not housed within a larger education 
entity. Instead, it is managed by the Office of Financial Management. The work of the ERDC is guided by three 
committees that consist of representatives from state-level agencies who share data through the System, 
stakeholder groups, and ERDC staff members. Each committee serves a different role in guiding the use and 
functionality of the System. The legislation that guides the work of the ERDC specifies a number of ways the 
partner agencies must collaborate, including identifying research questions, developing long-term plans to 
support transition across the continuum, and aiding in the development of an enrollment plan for higher 
education to meet workforce needs.°*° 
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Figure 11 | ERDC Data Request Process** 
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critical questions? contributor will review the request and may 
authorize ERDC to respond to the request. Is the requester able to sign a 
data sharing agreement and 
NOE abide by its terms, including 
data security provisions? 


= 
YES 


YES 


Yes, ERDC can 

respond to the ERDC will not 
data request with be able to fulfill 
aggregate or fully the data request. 
de-identified data 


Yes, ERDC can 
respond to the 


data request with 
individual-level data. 


Use and Impact 


Through close partnerships with nonpartisan legislative staff, as well as staff at the agencies, the ERDC works 
to anticipate the needs of legislators prior to the start of the legislative session, and prepares data sets accord- 
ingly. Additionally, a large number of research questions and corresponding data sets have been created, 
which allows for quick completion of requests. One staff member estimated that the Center receives up to six 
requests per week during legislative sessions. 


Next Steps 


The ERDC continues to work to increase the data sources available to the System and is currently working 

to include vocational rehabilitation training program data, TANF data, and criminal justice data from the 
Department of Corrections. The legislature recently instructed the Center to include juvenile justice data, which 
led to the creation of an operational data store at OFM’s Criminal Justice Statistical Analysis Center focused on 
criminal and juvenile justice data broadly. ERDC is also working to identify ways to include social services and 
healthcare data to help answer research questions about barriers to academic completion. 


CONNECTING THE CONTINUUM | LONGITUDINAL DATA SYSTEMS IN NORTH CAROLINA 


16 


Keys to Success 


A number of factors have contributed to the success of Washington’s Education Research and Data Center. 


> Strong support from the Governor and legislature has resulted in effective legislation and funding to 
enable the System to be built and continue meeting the needs of the state. Ongoing legislative funding 
also instills confidence in the Center’s long-term sustainability. 


By publishing up-to-date information about current requests to the System on their website and 

taking active steps to make the request process more transparent, the ERDC demonstrates a strong 
commitment to transparency. This helps build trust among the partners and the public. Additionally, 
the creation of publicly available data dashboards enables the public to see the type of work completed 
by the Center. 


After initially receiving only occasional requests, the Center actively worked to build and maintain a 
close partnership with the legislature. This partnership has resulted in meetings with nonpartisan staff 
to anticipate legislative needs in the upcoming session and allowed the Center to create data sets prior 
to the beginning of the legislative session to ensure information could be provided in a timely manner 
once the session began. 


> The ERDC is housed within a non-education entity. The ERDC does not manage any education programs 
and does not regulate any education institutions or agencies. While this was challenging initially, it has 
likely helped lead to strong support and collaboration with the various partner agencies as the agencies 
view the ERDC as objective and unbiased. 


Use of a centralized data system ensures that stakeholders consistently have access to high-quality 
data that can be retrieved in a reasonable amount of time. 


> The willingness of the Center to continue revising and improving based on best practices ensures 
that the System remains efficient, effective, and relevant, and that current data privacy and security 
standards are met. 


Challenges 


> The ERDC continues to work to improve data matching to support accurate and timely matches across 
data sources. The Center is in the process of adding information from state driver’s licenses which will 
help match data more accurately. Matching has also gotten better over the lifetime of the System, as 
there are now more data points from which to draw matching identifying information. 


> While the legislature has expanded funding for the ERDC and the ERDC has received almost $27 million 
in grants from the U.S. Department of Education and the U.S. Department of Labor, funding has not 
expanded as quickly as the demands on the Center have. Additional funding would help the Center 
meet increasing demand for its services and data products. 
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Guiding Questions for Policymakers 


As policymakers take a closer look at statewide longitudinal data systems, there are a number of questions 
they should consider when determining the effectiveness and functionality of the Systems. These questions 
may be helpful in guiding decision-making at the policy level. 


What questions do we need to answer in order to make progress toward our state’s 
education goals? 


Determining which questions can be answered by an SLDS and which cannot will require engagement 
among stakeholder groups, partners, and data analysts. These conversations are best held during the 
initial planning period of a system in order to frame and set expectations and inform the type of system 
that will best serve the needs of partners and stakeholders. Determining a research agenda in advance will 
also guide policymakers as they seek to identify what types of data will be needed, who possesses those 
data, and which partners should be included as the SLDS is built out. 


Which audiences will benefit from data provided through the System? 


Identifying who will be served by the data helps to inform what data are gathered, how the data can best 
be presented for each group, and what level of staffing support will be required in order to produce and 
present those data. Each of these audiences can benefit from a strong SLDS for different reasons. Parents 
can make informed decisions about their child’s education, researchers can conduct large-scale analyses 
to determine causal relationships between programs and their participants, and legislative analysts can 
determine which policies are producing the greatest benefits for students. 


What partners are engaged in data sharing? 


Individuals engaged in building state longitudinal data systems cite that mandating partner engagement 
through legislation is important in bringing entities to the table. However, legislation alone isn’t sufficient 
to build trusting relationships among partners. Partners should have clear expectations of what qualifies 
as timely submission of data and processes for data matching to ensure that the System is able to produce 
complete data sets that are meaningful. Clearly articulated memoranda of understanding (MOUs) play 

an important role in developing parity among partners. Partners need to be able to see the value in their 
participation and understand the benefits that their agencies will see from participation. 


Does the SLDS have a robust governance structure in place to provide necessary 
oversight and coordination? 


System governance is critical, and it requires trust and transparency among and between partners. 

The governing body serves a pivotal role in ensuring that the System is functional. Policymakers should 
identify whether a clear leadership structure has been established within the governance body to ensure 
that decisions are made efficiently. This structure can take many forms, but in North Carolina, a lack of 
clear governance may have contributed to the lengthy completion timeline. The governance structure will 
also play an important role in considering the benefits and challenges associated with both centralized and 
federated systems and determining whether a longitudinal data system should be a centralized system or 
federated system. 
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Guiding Questions for Policymakers (cont.) 


What are the staffing and technological requirements needed to reliably collect, 
maintain, and submit data? What personnel are needed to ensure effective 
governance and communication? 


Teams working with state longitudinal data systems should include data analysts who are experienced in 
analyzing education and workforce data. Experienced education data analysts will ensure that data are 
analyzed accurately and appropriately so that the data can be used and presented with the appropriate 
context. This capacity needs to be built both within each member agency and within the coordinating 
agency. Building the requisite technological ability is costly but critical. Similarly, human capital will need 
to be added and strengthened at the institution level in order to reliably collect and report data to, and 
across, systems. 


How is the System funded, and is that funding sustainable? 


The large startup investments made by the federal government enabled many states, including North 
Carolina, to build systems that otherwise may not have been created. However, many states have not 
created sustainable funding streams to maintain and improve these systems. In order to ensure the long- 
term sustainability and success of state longitudinal data systems, there should be permanent or recurring 
state funding. These funds should be provided not only for the governing body, but also to ensure that 
there are sufficient funds at each partner entity to maintain the needed technology infrastructure, as 

well as human capacity both in IT and data analysis. Beyond funding of the overall system, it is critical that 
the technology infrastructure of each partner is sufficient in order to provide timely, accurate data to the 
System. This will require varying investments in each partner entity. 


How will stakeholders access the information in the System? 


For longitudinal data systems to truly be functional, data usability must be taken into account from the 
beginning. In addition to sharing data with outside researchers, some states have chosen to build this 
capacity at a central coordinating agency that regularly builds reports and/or interactive dashboards that 
can be easily used and understood by policymakers and the general public. 


What steps are being taken to ensure that data privacy is protected? 


Public trust and confidence in the System is dependent on the knowledge that personally identifiable 

data cannot be accessed or used to connect an individual to their education and workforce history. 
Policies must be enacted that span physical, technological, and legal data protections. The governing body 
plays a critical role in ensuring data are securely held and privacy efforts are transparent to partners, 
stakeholders, and the community. 
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Glossary 


Section 504 of the Rehabilitation Act 


504 of 1973 KDE Kentucky Department of Education 
AP Advanced Placement KLDS Kentucky Longitudinal Data System 
American Recovery and er 
ARRA Ralnuiestinent Actor 2000 KYSTATS Kentucky Center for Statistics 
North Carolina’s Common Education . 
CEDARS Data Analysis and Reperting System LEA Local Education Agency 
CEDS Common Education Data Standards mCLASS | ee erine 200! EAE Ne ae =e 
development of reading skills 
CFS Roni Carolinas Common Follow-up MOU Memorandum of Understanding 
System 
CTE Career and Technical Education NCCCS Bert Cargind Community Colces 
System 
ECE Early Childhood Education NCDHHS North Carolina Sparen of Health 
and Human Services 
ECIDS Early Childhood Integrated Data NCDPI North Carolina Department of Public 
System Instruction 
North Carolina Education North Carolina Early Childhood 
me Longitudinal Data System ta Integrated Data System 
EOC North Carolina’s End of Course Test NCICU Maren Carolina independent Colleges 
and Universities 
EOG North Carolina’s End of Grade Test OFM Se aN Sateen iments 
Management 
Washington’s Education Research : : 
ERDC Sace Dana Center P-20 pre-K through Higher Education 
Family Educational Rights and ! pre-K through Higher Education and 
sata Privacy Act of 1974 on the Workforce 
FRL Free/Reduced Lunch Status SES Socioeconomic Status 
GDAC Orn Carolina 5 Gavernment Bate SLDS State Longitudinal Data System 
Analytics Center 
IB International Baccalaureate Program SNAP Supplemental: Nucimoninasistante 
Program 
IEP Individualized Education Program TANF Temporary Assistance:for Needy 
Families 
IES U.S. Department of Education UERS North Carolina’s Uniform Education 
Institute of Education Sciences Reporting System 
IRB Institutional Review Board UID Unique Identifier 
IT Information Technology UNC University of North Carolina System 
KCEWS Kentucky Center for Education and WwDQl Workforce Data Quality Initiative 


Workforce Statistics 
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